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Tlie appropriate description of fluctuations witliin the framework of evolutionary game theory 
is a fundamental unsolved problem in the case of finite populations. The Moran process recently 
introduced into this context in Nowak et at, [Nature (London) 428, 646 (2004)] defines a promising 
standard model of evolutionary game theory in finite populations for which analytical results are 
accessible. In this paper, we derive the stationary distribution of the Moran process population 
dynamics for arbitrary 2x2 games for the finite size case. We show that a nonvanishing background 
fitness can be transformed to the vanishing case by rescaling the payoff matrix. In contrast to the 
common approach to mimic finite-size fluctuations by Gaussian distributed noise, the finite-size 
fiuctuations can deviate significantly from a Gaussian distribution. 
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Theoretical studies of coevolutionary dynamics usuaUy 
assume infinite populations, as the replicator dynamics 
P, Q or the Lotka-Volterra equations 0, Q ■ The limit of 
infinite populations leading to deterministic differential 
equations is an idealization motivated mainly by math- 
ematical convenience. Only in few cases the population 
will be large enough to justify the assumption of infinite 
populations. 

In finite populations, crucial differences can appear. 
Population states that cannot be invaded by a small frac- 
tion of mutants in infinite population, so-called Evolu- 
tionary Stable Strategies 0, can be invaded by a single 
mutant Q . In addition, a certain inherent stochasticity is 
always present in finite populations. In multipopulation 
interactions, such fluctuations can possibly be exploited 
0. In this paper, we quantify the inherent fluctuations 
arising from finite populations. As a starting point, we 
investigate the classical Moran process that was re- 
cently transfered to frequency dependent selection ^ Q . 
In a Moran process, in each time step one agent is repli- 
cated and one agent is eliminated. Thus the total size 
of the population is strictly conserved. This process can 
be considered as a standard model for game dynamics in 
finite populations. Although a strictly fixed population 
size will be fulfilled only in systems with hard resource 
limitations, e.g. a fixed number of academic positions, it 
is a widely common default, especially in spatial games 
0, 0, ^1 Q • From a systematic point of view, the 
dynamics within this process and the nature of the fluc- 
tuations have to be understood before a generalization to 
variable population sizes on solid grounds is possible. 

In we have shown that the Moran process intro- 
duced in can be derived as a mean-field approximation 
of the finite population game dynamics. In mean-field 
theories of evolutionary game theory [l^ |[l6|, [T^, ll^ not 
only the spatial degrees of freedom are neglected; but 
the limit of infinite populations also implies a transition 
from a stochastic system to a deterministic equation of 
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motion. While the average effect of mutations can of- 
ten be lumped in a deterministic term (f 5> ,21] , different 
ways to incorporate external stochasticity have been pro- 
posed, e.g. by a Langevin term of Gaussian distributed 
noise ^UJ,^ or stochastic payoffs 22]. Consequently, 
one could also approximate the intrinsic noise of the finite 
system by Gaussian noise reintroduced into the contin- 
uum equations. But a prion it is not clear, in which 
situation this approximation is justified. Especially in 
small populations, the inherent stochasticity may signif- 
icantly exceed any external noise. In a finite-round Pris- 
oner's Dilemma game, the broadness of the distribution 
of cooperators recently was found to promote coopera- 
tion [23 ■ Further, the distribution decay of fluctuations 
is known to be of substantial impact both in genetic evo- 
lutionary dynamics 24] and in evolutionary optimization 

To clarify the nature of inherent fluctuations of evo- 
lutionary dynamics in a Moran process is the scope of 
this paper. We quantify the deviations from the mean 
value by explicitly calculating the stationary distribu- 
tion of strategies for general 2x2 games and provide a 
transformation for the case of nonvanishing background 
fitness. The process is illustrated with two qualitatively 
representative kinds of games, and the exact solution, 
also for the more realistic situation of a nonvanishing 
background fitness, is provided. 

Moran evolution dynamics in 2x2 games. — We con- 
sider a finite population of N agents of two different types, 
A and B, interacting in a game with the payoff matrix 



P = 



a b 
c d 



(1) 



Each agent interacts with a certain number of randomly 
chosen partners. The A individual s obtains the fitness 



= 1 



A 



(2) 



•Electronic address: 
t Electronic address: 



claussen@theo-physik.unr 



traiHsen^the^^l^sOrun^ierde] 



where 



n'/ (f^s ) is the number of interactions with A (B) 
individuals. < w < f measures the contribution of 
the game to the fitness, f — w is the background fitness. 



2 



An equivalent equation holds for B agents. Occasion- 
ally, the payoff of a randomly chosen individual s is com- 
pared with the payoff of another randomly chosen agent 
u. With probability tTs/{t:s + t^u), a copy of agent s re- 
places agent u. With probability 7r„/(7rs -|- 7r„), agent s 
is replaced by a copy of u. The probability that an agent 
reproduces is hence proportional to its payoff. The pay- 
off depends on the type of the individual and on the kind 
of its interactions. This approach is fre quently used in 
simulations of multiagent systems |2^|23,|2l,|23, genetic 
algorithms js^, |^ , and evolutionary game theory . 

The averaged dynamics of this model can be computed 
from a mean- field theory If every agent interacts 

with a representative sample of the population, the aver- 
age payoff of A and B individuals will be, respectively, 

a{t-l) + b{N-z) 

L — W + W — ; [6) 



7r"^(i) = 1 — w + w 



N-1 

ci + d{N - 1 - i) 
N - 1 ' 



where i is the number of A individuals. We explicitly 
excluded self interactions. An individual is selected for 
reproduction with a probability proportional to its pay- 
off, as described above. It replaces an individual that is 
chosen at random. This reduces the process to a Moran 
process Q , which was recently transfered to a game the- 
oretic context 0] . The corresponding mean- field dy- 
namics is given by a Markov process with the transition 
probabilities [l^ 



T ■ 



N -i 



(4) 



n^ii) (N-i) 



Tr^{i)i + Tri^{i) {N-i)N 
1 — Ti^i^i — Ti^i-i. 



All other transition probabilities are zero. The states 
i = Q and i — N are absorbing, while the remaining 
states are transient. Conveniently, a small mutation can 
be introduced to allow for an escape from the absorbing 
states js^. 

The general case of nonvanishing background fitness. 
For a nonvanishing background fitness 1— w > the tran- 
sition properties obtained directly from Eqs. lO and Q 
become quite lengthy. A more elegant way is to rescale 
the payoff matrix of a given 2x2 game according to 



a' b' 
c' d' 



1 + (a - l)w 
1 + (c- l)w 



1 + (5- l)w 
l + {d- l)w 



(5) 



With this rescaled payoff matrix, a vanishing background 
fitness can be assumed in Q without loss of generality. 

Fluctuations around the average strategy: In order to 
quantify the deviations from the average strategy of the 
system, we compute the stationary distribution Pi for 
this system. We assume a small mutation probability fi. 
For /i ^ 1, mutations affect the system only in the states 
that are absorbing for /i = 0. In this case, the strategy 
distribution is generated only by the inherent stochastic- 
ity of the finite population. The stationary probability 
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FIG. 1: Stationary probability distribution for different evolu- 
tionary dynamics depending on the distance to the maximum 
(N=100). For comparison, also the slow decay for neutral evo- 
lution is shown. The decay of the distribution can be fitted 
by a stretched exponential exp(— foa;^) with 7 = 2.06 (antico- 
ordination game), 7 = 0.87 (constant fitness), and 7 = 0.63 
(Prisoner's Dilemma). The inset shows the same data where 
both axes are logarithmized, thus stretched exponentials ap- 
pear as straight lines. The decay deviates significantly from 
a Gaussian distribution for constant fitness and Prisoner's 
Dilemma, corresponding to a random motion in an anhar- 
monic potential. 



can be computed in the interior independently from the 
boundaries, the correct normalization can then be found 
analyzing the transitions from the boundaries to the in- 
terior, i.e. PoM — PiTi-to- 

Let us first consider the neutral evolution limit of w — 
0, where the fitness is constant and independent of the 
type. The payoffs are 7r'^(i) = Tr^{i) = 1. This implies 



+1+1 



i{N -i) 



(6) 



From Pi Ti- 
0<i<N 



^1 = Pi+i Ti^i^i we find in equilibrium for 



P oc 



(7) 



which has a minimum aX i — N/2. The equilibrium dis- 
tribution arises from a neutral evolution of two types, as 
known from population genetics ^33] . 

Constant fitness. The simplest case for w > is the 
case of constant fitness, i.e., a = b < c = d = 1. The 
evolutionary dynamics drifts towards the type B, which 
has higher fitness. We find for the stationary probability 
distribution (0 < i < - 1) 



P. 



P 



_i r(i + l) + N-i-l 
= r 



N-i 



N - 



i + lN ~i-l' 



(8) 



where r = I — w + wa < 1. Far from the borders (at i = 
0,N), Pi^i/Pi converges to r implying an exponential 
decay of the stationary probability distribution. 
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FIG. 2: Scaling of the variance, normalized by A'^, of the finite- 
size fluctuations for anticoordination game (slope —1/2), con- 
stant fitness (slope —1), and Prisoner's dilemma (slope —3/2). 
For neutral evolution (not shown) the variance increases faster 
than N. 

Internal Nash equilibrium. For frequency dependent 
fitness and w > 0, the game can have an internal Nash 
equilibrium or an equilibrium in one of the absorbing 
states. 

As a simple example with an internal Nash equilibrium 
we choose a simple "anticoordination" game with w — I, 



P 



1 

1 



For the transition probabilities, we find 

N -i 



2N 
i 

2iV' 



(9) 



(10) 



which describes a random walk with a drift towards the 
deterministic fixed point i = N/2. In equilibrium, we 
have Pi Ti^i+i = Pj+i Tj+i^j for every i, which leads to 



3=0 



N 
i + 1 



(11) 



where Pq is determined by normalization. Pi is a bino- 
mial distribution around the equilibrium of the replicator 

-N (N\ 

i • 



dynamics at i = N/2, Pi = 2"^ (^). 



Prisoner's Dilemma: Nash equilibrium at the border. 
The Prisoner's Dilemma 34] is a standard model, where 
mutual cooperation leads to highest payoff in the iter- 
ated game. It is motivated by the situation where two 
prisoners can reduce their time in prison by witnessing 
the other's guilt ("defect"). On the other hand, if both 
"cooperate" and refrain from blaming the other, both re- 
ceive a reduction of punishment. This is described with 
parameters fulfilling c > a > d > b; the dilemma sit- 
uation originates from the temptation c > a, defection 
yields a higher payoff if the opponent cooperates. In its 
standard parameters, the Prisoner's Dilemma is defined 
by the payoff matrix 



P = 



3 
5 1 



(12) 



which has a Nash equilibrium for mutual defection, i.e. 
i — 0. As 6 = 0, also state i = 1 is absorbing for w = 1 
(two cooperators are needed to promote cooperation). 
Thus a small mutation rate /i has to be assumed also 
for Ti^2- Alternatively one could assume w < 1. The 
transition probabilities are given by 



Ti^i+i 
Ti^i-i 



3i-3 



iiN -i) 



-i^ -2i + 3iN + N{N -I) N 

Ai + N-1 i{N-i) 
-i^-2i + 3iN + N{N-l) N 



(13) 



From this, a closed form of the probability distribution 
can be derived (see below for a derivation with arbitrary 
payoff matrix) . A comparison between different station- 
ary distributions is shown in Fig.^ The finite-size scaling 
of the variance is shown for the same cases in Fig. 12 

Stationary Distribution for an arbitrary payoff matrix. 
For the ratio of the transition probabilities between i and 
« + 1 we find with w = 1, cf. Eq. 



T,^^+l _ 7r-^(i) (^ + 1)t:^{i + I) + [N - i - 1)7:^ {i + 1) i{N - i) 

T,+ l^,^ iTT'^{i) + {N ~i)TTB{i) 7r^(i + l) {i + l){N-i-l) 

a{i - 1) + b{N - i) i{N - i)[{i + if {a -b-c + d) + {i + l){-a + bN + cN + d - 2dN) + N{N - l)d] 
~ c{i + 1) + d{N - i - 2) {i + 1){N - i - l)[i2(a -b-c + d) + i{-a + bN + cN + d - 2dN) + N{N - l)d] 
_ a-b i-N^ i{N~i) (^ - N^){i - N^,) 

c-d i-Na {i + l){N -i-1) {i-N2){i-Niy ^ ' 

Here TVi • • • N^ are the roots of the quadratic expressions in i and A^5 = , Nq — ■ We have excluded the 

special cases a — 6 = 0, c — d = Q discussed above in and (a — 6)/(c — d) = 1, where some factors do not depend 
on i and part of the expression simplifies. For N — \ >k>i>\, the density of the stationary state can be solved 



4 



explicitly giving rising factorials (Pochhammer symbols), or equivalently, quotients of Gamma functions, 

Pk ^ 'yj' Tw^+i ^ (<^\ jjN-j) r(fc - N5)T{j - Ne)T{k - Ni)T{j - N2)T{k ~ N3)T{j - N^) 
P, yr,+i^, \c-dj ' kiN - k) ' T{j - Nr^)rik - Ne)rij - N,)rik ~ N2)rij - Ns)T{k - N^) 



which yields, after calculating Pm/Pn-i and Pi/Pq ex- 
plicitly, and after normalization, the total density of the 
stationary state. Equations lO and ((T3|l cover the general 
case of 2x2 games including nonvanishing background 
fitness. The previously discussed examples are included 
as special cases. 

To conclude, the distribution of the fluctuations 
around a Nash equilibrium can be nontrivially broad- 
ened in realistic models of evolutionary game theory. We 
analyzed the effect of internal noise stemming from the 
inherent evolutionary update fluctuations in a finite pop- 
ulation. In general, internal noise and externally imposed 
stochastic forces can follow qualitatively different distri- 
butions. In our paper, we concentrated on the impor- 
tant case of a Moran process, which can be considered 
as a standard model of evolutionary game dynamics in 



finite populations. For the Moran process, the effect of 
the finite size of the population can be accessed directly. 
Neglecting external noise, we have shown that the sta- 
tionary distribution of the Moran process of evolutionary 
2x2 games can be calculated analytically and yields dif- 
ferent decay tails of the distributions. Depending on the 
payoff matrix and the location of the Nash equilibrium, 
the finite size fluctuations may deviate significantly from 
a Gaussian distribution. 
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